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ABSTRACT.  The  Lanczos  algorithm  generates  Ritz  values  in  order  to 
approximate  eigenvalues.  If  some  eigenvalues  are  clustered  then  a  Ritz 
value  may  hover  at  a  wrong  value  for  a  good  number  of  steps.  We  study 
this  phenomenon  and  focus  on  the  point  of  discovery,  the  first  step  at 
which  it  is  certain  that  there  is  a  hidden  eigenvalue  in  the  vicinity  of 
stabilized  Ritz  values.  Both  before  and  after  this  point  the  Ritz  value 
behavior  is  routine  —  but  for  different  eigenvalue  configurations. 

The  "effective  spread”  at  step  j  is  an  interval  guaranteed  to 
contain  all  unknown  eigenvalues.  The  notion  of  "Ritz  intervals”  leads 
to  a  computable  counterpart  to  the  exact  theory.  . 
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1.  INTRODUCTION 


When  the  Lanczos  algorithm  is  applied  to  a  symmetric  n*n  matrix  A  it 

produces,  at  step  j,  an  unreduced  tridiagonal  matrix  T.  whose  eigen- 

J 

values 


e(j)  <  0(j)  <  <  e(j) 

12  J 

will  be  called  the  j  ^  set  of  Ritz  values.  The  final  set  of  Ritz  values 
are  the  eigenvalues  of  A, 


A  =  0(n\...,A  = 

i  i  *  *  n  n 


We  may  assume,  without  loss  of  generality,  that 
from  each  other.  The  characteristic  polynomial 


the  {A^}  are  all  distinct 

of  T .  is 
J 


*j(() 


rr  (f  -  e{ih 

i=l  1 


One  attraction  of  the  Lanczos  algorithm  is  that  in  most  cases  for  small 
values  of  j,  but  j  i  max {10,  Jn] ,  same  extreme  Ritz  values  from  the 
Set  are  very  good  approximations  to  same  extreme  eigenvalues  of  A. 
Error  bounds  on  -  A^J  were  given  in  (Kaniel  1966;  Paige  1972; 

Saad  1980).  Here  we  concentrate  on  a  related  question  concerning 
occasional  eccentric  behavior  of  the  Ritz  values.  When  a  cluster  of 
(unknown)  eigenvalues  has  a  spread  that  is  1*  or  less  of  neighboring 
eigenvalue  gaps  then  a  Ritz  value  can  hover  at  the  mean  of  the  cluster 
for  8  or  more  successive  values  of  j  before  making  an  abrupt  shift  to 
one  of  the  eigenvalues.  This  phenomenon  is  called  miacoaverg&oce  (see 
Parlett  et  al.,  1982).  It  was  studied  very  carefully  in  (van  der  Sluis 
and  van  der  Vorst  1987)  and  we  shall  refer  frequently  to  this  paper. 

We  offer  a  simple  description  of  this  odd  Ritz  value  behavior  that 
complements  the  detailed  error  bounds  in  (S.  and  V.  1987).  We  find  it 
necessary  to  look  beyond  the  Ritz  values  near  the  cluster  for  a  simple 
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explanation  and  are  lead  to  the  effective  spread  which  gives  the  allow¬ 
able  range  for  the  unknown  eigenvalues  at  each  step.  This  concept  is 
useful  in  describing  the  actual  convergence  rate  of  the  conjugate 
gradient  algorithm. 

After  some  preliminaries  the  paper  is  organized  as  follows.  The 
example  in  (S.  and  V.  1987)  and  a  snapshot  of  the  paper's  many  results 
is  followed  by  our  formal  description  in  Section  5.  Then,  by  using 
Paige's  persistence  theorem,  we  can  offer  a  constructive  theory  that 
avoids  reference  to  unknown  quantities.  Finally  we  present  a  theorem 
which  shows  that  the  special  value  of  j  at  which  the  presence  of  a 
hidden  eigenvalue  is  detected  (the  point  of  discovery)  can  be  character¬ 
ized  by  the  occurrence  of  a  local  minimum  in  the  error  bound  of  the 
misleading  stagnant  Ritz  value. 

The  figures  and  charts  are  worth  a  thousand  words. 

2.  ORTHOGONAL  POLYNOMIALS 

By  Cauchy’s  interlace  theorem  (Parlett  1980,  Chap.  10  or  S.  and  V.  1987) 
each  set  of  Ritz  values  interlaces  the  next  set.  Moreover,  the  charac¬ 
teristic  polynomials  are  orthogonal  with  respect  to  a  special  discrete 
inner  product  function 

n 

<♦,*->  :=  l  w.*(X.)V(A.)  . 

k=l  1  1  1 

Bach  positive  weight  {w^}  is  the  squared  cosine  of  the  angle  between  the 

\tb  eigenvector  of  A  and  the  starting  vector  of  the  Lanczos  algorithm. 

Thus  the  whole  set  of  Ritz  values  is  determined  by  the  inner  product  and 

this  point  of  view  is  the  fundamental  one.  It  follows  from  orthogonality 

that  each  x •  i®  minimal  (Szego  1939): 

J 

lx J  =  ■in  !♦! 

J  ♦ 
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over  all  sonic  polynomials  ♦  of  decree  not  exceeding  j.  Here 

1+9  :=  <♦.♦>*  . 

The  drawback  of  this  approach  is  that  in  the  Lanczos  context  <  ,  >  is 

unknown  and  the  x  •  are  built  up  one  at  a  time. 

J 

We  shall  employ  both  viewpoints  in  this  paper. 

3.  THE  DUTCH  EXAMPLE 

The  following  values  are  used  in  (S.  and  V.  1987). 


X.  = 

.0340  , 

Xa  =  .0341  , 

X3  =  .0820  , 

K  = 

.127  , 

X5  =  .155  , 

X4  =  .190  , 

X?,...  A>oo  uniformly  spaced  in  [.2,  1.2]. 
wA  =  1/900,  i  =  1,...,900  . 

It  should  be  emphasized  that  and  X^  are  not  to  be  considered 

pathologically  close.  The  gap  poses  no  difficulties  to  current  software 

when  invoked  using  numbers  with  at  least  14  decimal  digits.  The  question 

here  is  how  many  more  steps  does  the  Lanczos  algorithm  require  to  find 

X  and  X  than  to  find  X  ? 
i  a  s 

Figure  1  shows  those  Ritz  values  that  are  near  Xt  and  X^  for 
j  =  7 , . . . ,  30 .  For  j  =  20 , . . . ,  25 

.03404952  <  0^J)  <  .03404999  , 

so  the  changes  during  this  stagnation  phase  are  smaller  than  1%  of  the 
actual  error,  0^  -  X  . 

In  (S.  and  V.  1987)  the  behavior  of  0^  is  divided  into  three 
phases : 

i)  normal  (rapid)  convergence  of  0 ^  to  the  mean  (Xi+Xj)/2, 

j  ~  1, . . . , 18. 

ii)  stagnation  at  the  mean,  j  =  19,..., 27. 
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iii)  normal  (rapid)  convergence  to  X^  in  step  with  normal  (rapid) 
convergence  of  0^^  to  Xjt  j  =  28,... ,33. 

4.  ERROR  BOUNDS 

The  Kaniel-Paige-Saad  error  bounds,  see  (Parlett  1980,  Chap.  12),  give 

upper  bounds  on  the  "errors"  0^  -  Xjf  k  =  1,2,...  in  terms  of 

limited  information  about  the  other  eigenvalues.  Here  we  mention  that 

each  polynomial  of  degree  j  yields  an  upper  bound  on  0^  -  X^ 

(and  X  -  and,  by  specializing  the  choice,  one  can  obtain  bounds 

n  j 

on  -  X^  for  higher  values  of  k. 

In  (S.  and  V.  1987)  a  number  of  theorems  are  proved  that  yield 
expressions  for  the  particular  steps  in  a  Lanczos  run  that  separate  the 
three  phases  exhibited  in  the  numerical  example.  The  analysis  uses 
several  nearby  inner  products  and  their  associated  Ritz  values  for 
comparison  purposes.  The  natural  comparison  is  the  inner  product  in 
which  Xt  and  Xg  are  replaced  by 

M  :=  S4(Xt  +  Xa) 

and  the  weight  is  adjusted  according  to  w^  =  •J~2.  The  associated  Ritz 
values  are  denoted  by  0^\  i  =  l,...,j. 

We  quote  one  result  that  determines  for  how  many  steps  0^  remains 
very  close  behind  to  0^. 


Property  8.17.  Assume 

0  <  £  <  %  and 

i+l 

(W  1  3rV 

k=3  k  » 

*  l 

(8.18) 

If,  for  p  €  (0,  1— e ) , 

0(l)  -  M  i  (X  -X  ) 
i  a  i ' 

t/2  p  , 

(8.20) 

then 

e[ -  M 

1  *  *— “"TTT -  * 

5(l)  -  N 

i 

1-t-p 

(8.21) 

When  c  and  p  have  been  chosen  appropriately  the  theorem  says 
that  it  is  the  eventual  violation  of  (8.20)  that  breaks  8^’s  bondage 
to  0^.  In  the  example  e  <  .0131  and  so  p  =  0.2  is  a  valid 
choice  that  yields  a  value  0.33  *  10  s  for  the  right  side  of  (8.20). 

This  is  violated  (barely)  at  Step  18  which  Fig.  1  reveals  as  the 
end  of  Phase  1. 

Different  theorems  are  used  to  characterize  the  duration  of  the 
stagnation  phase  and  the  final  convergence  of  0^  and  0^>  to 


X  and  X  . 
1  2 


5.  THE  POINT  OF  DISCOVERY 

The  many  quantitative  results  in  (S.  and  V.  1987)  give  a  very  accurate 
description  of  the  behavior  of  0^  and  0^\  as  j  increases,  in 
terms  of  the  eigenvalues.  However  two  distinct  aspects  of  the  process 
are  fused  in  those  theorems:  the  routine  and  the  exceptional.  We 
suggest  an  alternative  description  based  on  a  single  whole  number,  the 
step  at  which  the  comparison  scheme  must  be  discarded. 
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There  is  an  index  j,  the  point  of  discovery,  at  which 
the  process  "perceives"  that  there  must  be  a  hidden 
eigenvalue  in  the  vicinity  of  the  stagnant  Ritz  values. 

Prior  to  step  j  the  Ritz  value  development  is  routine  — 
but  for  the  comparison  scheme  wherein  the  close  eigenvalues 
are  pushed  to  their  mean  value.  After  step  j  the 
development  is  also  routine  —  for  the  true  regime. 

In  the  Dutch  example  the  point  of  discovery  is  step  24.  However 
this  fact  cannot  be  deduced  from  a  local  analysis  of  0^  or  even  of 
and  0^ .  Such  a  specific  naming  of  the  discovery  point 
is  an  artefact  of  our  definition  as  well  as  of  the  situation  itself. 

Our  choice,  given  in  Section  8,  has  the  attractive  feature  that  no 
knowledge  of  the  true  eigenvalues  is  needed  to  determine  it.  A  simpler 
definition  of  the  discovery  point,  arising  from  notions  to  be  given  in 
this  section,  would  put  the  discovery  at  step  22  but  this  approach 
requires  knowledge  of  the  eigenvalues  to  high  accuracy. 

We  mention  here  that  it  is  only  after  step  24  that  approximates 

Aj'  for  k  =  1,2, 3, 4  and  5.  Before  that  there  is  some  mismatch  between  the 
indices  of  Ritz  values  and  of  the  eigenvalues  they  approximate.  When  we 
say  that  the  development  is  routine  we  mean  that  the  Kaniel-Paige-Saad 
error  bounds  are  quite  sufficient  to  describe  how  well  certain  eigen¬ 
values  are  approximated  at  each  (routine)  step. 

In  order  to  justify  this  description  of  Ritz  value  behavior  some 
preparation  is  necessary. 

Figure  2  shows  the  positions  of  the  smaller  Ritz  values  for  the 
comparison  inner  product  (wherein  A{  and  A^  are  moved  to  their  mean). 
The  horizontal  bars  denote  intervals  guaranteed  to  contain  an  eigenvalue. 
The  configuration  within  the  crowded  interval  [.2,  1.2],  where  894  eigen¬ 
values  are  located,  is  of  little  direct  concern  in  this  study  and  it  is 
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sot  shown.  The  pattern  revealed  in  Figure  2  should  be  considered  routine 
for  all  steps.  The  gaps  between  the  six  saallest  eigenvalues  are  fairly 
uniform. 

Recall  that  {*  ,}D  denotes  the  sequence  of  orthogonal  polynomials 
J  * 

associated  with  an  inner  product. 

DEFINITION.  For  a  given  inner  product  and  any  (  E  R, 

•  Left(j;  f)  :=  the  number  of  zeros  of  * .  that  are  less  than  (. 

J 

•  Right(j;  ()  :=  the  number  of  zeros  of  *.  that  exceed  (. 

J 


LEf-MA  1.  For  a  fixed  f  both  Left(j;()  and  Right(j;f)  are 
monotone  nondecreasing  in  j . 

This  result  is  a  direct  consequence  of  the  interlacing  property. 

In  fact,  for  all  j, 

0  <  Left( j+1;  ()  -  Left( j;  f)  <  1  . 

From  now  on  we  confine  the  discussion  to  discrete  inner  products  with 
n  points  of  increase.  What  is  of  interest  is  the  smallest  index  i  at 
which  Left(i;  ()  attains  its  maximum  value  Left(n;  (),  the  number  of 
eigenvalues  less  than  (. 

Figure  2  reveals  that  at  step  18  the  three  smallest  eigenvalues 
are  marked  by  the  three  smallest  Ritz  values  while  >  A?.  This 

suggests  the  following  definitions: 

[Note  that  Left(k,  !4(A.+A.+1))  <  i  =  Left(n,  *(*i+Xi+l))] 


L( j)  :=  min{i:  Left(j;  !4(Ai+A.+1>)  <  i}  , 

R(j)  :=  min{i:  Right(j,  54(An  i+An  i+1))  <  i}  . 

The  effective  spread  at  step  j  :=  A^^  ~  ^R(j)  ' 
The  eigenvalues  outside  [A^jj,  ^R(j)^  are 


The  eigenvalues  outside  [A 
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marked  (at  step  j). 

•  The  effective  condition  number  for  the  assocaited  Conjugate 

Gradient  Algorithm  (when  0  <  A|  <  ...  <  An)  is 

XR(j)/XL(j)  * 

The  following  result  gives  substance  to  the  words  "marked”  and 
"spread"  used  above. 


LEMMA  2.  for  each  k  £  j  (but  k  <  nj, 

ejk)  6  [A.,  ,  for  i  =  1,2, .. . ,L( j)-l  , 

e  vwi  •  1  =  !>2 . • 

0ik)  6  [XL(j)’  XR(j)]  *  for  L(j)  *  1  *  k+1“R(j)  * 


Proof.  The  first  two  assertions  are  direct  consequences  of  the 
interlace  property  and  the  definition  of  L(j)  and  R(j).  Recall  that 


The  third  assertion  is  seen  by  contradiction.  Suppose 
for  some  k  *  j,  then 


8(k)  <  A 

®L(j)  XL(j) 


Left(k,  XL(j))  A  L( j)  >  L(j)-1  =  Left(n,  A1(J))  . 

The  first  inequality  follows  from  the  definition  of  Left,  the  final 
equality  follows  from  the  definition  of  L. 

This  inequality  contradicts  the  fact  that  Left(j,  ()  is  monotone 

(k) 

nondecreasing  in  j  (i.e.  Lemma  1).  Similarly  cannot 

exceed  AR^  for  any  k  A  j.  □ 


Without  knowledge  of  the  {A^}  the  indices  L(j)  and  R(j)  will 
also  be  unknown.  We  provide  a  computable  alternative  in  Section  8. 

Here  we  seek  only  to  explain  Ritz  value  behavior.  In  order  to  quantify 
the  late  recognition  of  eigenvalue  clusters  we  need  one  more  elementary 
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consequence  of  the  interlacing  property.  L(j)  is  the  index  of  the 
smallest  unmarked  eigenvalue.  However  it  can  happen  that 
Left(j,  !*(Xj+X^+^))  =  i  for  some  i  >  L(j).  In  fact,  whenever  the 
largest  index  i  for  which  Left(j,  i4(X^+X^+j))  achieves  its  maximum 
exceeds  L(j),  then  the  existence  of  hidden  eigenvalues  has  been 
discovered.  This  relatively  uncommon  event  occurs  in  the  Dutch  example. 
It  can  be  seen  at  step  24  in  Figure  3,  which  shows  all  the  Ritz  values 
less  than  0.2. 

To  formalize  our  explanation  we  make  one  more  pair  of  definitions: 


L(j) 


max{i:  Left(j,  54(Xi  +  X.+1))  =  i}  , 
0  ,  if  the  set  is  empty 


R(j) 


max{i:  Right (j,  '4(Xn+1_i  +  X^))  =  i}  , 
0  ,  if  the  set  is  empty 


•  The  (theoretical)  point  of  Discovery 

:=  min{j:  L(j)  >  L(j)  or  R(j)  >  R(j)}  . 

We  can  call  the  configuration  of  Ritz  values  routine  whenever 

L(j)  +  1  =  L(j)  ,  R(j)  +  1  =  R(j) 


6.  DISCUSSION  OF  THE  DUTCH  EXAMPLE 

Figure  3  reveals  that,  at  step  22,  9 ~  Xa  and  9 s  *  X^.  There  is 
every  indication  that  this  is  a  routine  case.  Comparison  with  Figure  2 
(the  comparison  scheme)  indicates  that  to  within  human  vision  the  two 

sets  of  Ritz  values  are  identical  through  step  20.  In  the  absence  of 

(k) 

outside  information  one  assumes  that  9 is  going  to  converge  to  an 
eigenvalue  near  0.12.  However  a  careful  inspection  of  the  computed  Ritz 


n 


(22} 

values  show  that  0'  -  0.0819858  <  X  =  0.082.  By  Lemma  1, 

a  3 

Left(k,3)  £  2  for  all  k  >  22  and  so  there  must  be  a  hidden  eigen¬ 
value.  So  step  22  has  a  good  claim  to  be  the  point  of  discovery. 

Such  a  definition  demands  full  knowledge  of  the  configuration  in  which 
case  the  existence  of  Xa  will  already  be  known.  Our  interest  comes 
from  the  practical  situation  where  we  want  to  make  inferences  about  the 
unknown  eigenvalues  from  the  known  Ritz  values. 

At  step  23  8*  slips  noticeably  below  X$.  However  it  is  not 

until  step  24  that  8  is  closer  to  X  than  to  X  and  so  8  marks 

4  4  S  4 

X  before  8a  marks  X^  at  step  26.  Table  1  reiterates  the  message  of 

Figure  3  but  does  so  by  means  of  natural  numbers.  Moreover,  the  profile 

vector  in  the  table  shows  the  claim  of  step  22  to  be  the  point  of 

discovery.  Our  use  of  midpoints  between  eigenvalues  in  the  definition 
*  * 

of  L  and  L  accounts  for  the  fact  that  L(j)  does  not  exceed  L(j) 

until  step  24.  This  value  is  corroborated  in  Section  8.  The  reward  for 

using  midpoints  is  that,  for  i  <  L(j),  does  actually  mark  X^. 

The  detailed  results  in  (S.  and  V.  1987)  do  not  reveal  that  at  step 

23  the  close  pair  of  eigenvalues  near  .034  caused  the  far  off  sequence 
(k) 

{<r  }  to  be  deflected  from  its  apparent  target  X4  =  .127  towards  the 

(k) 

hidden  pair  and  the  concomittant  deflection  of  {(r  }  to  X^.  This  is 

the  essential  insight  provided  by  Figure  3  which  also  shows  how  mislead- 

(k)  (k) 

ing  it  can  be  to  automatically  associate  8 )  with  X^  or 

with  There  is  no  natural  pairing  of  Ritz  values  at  adjacent 

steps.  We  offer  our  preferred  way  of  pairing  in  Section  7. 
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Profile ( j) 


Table  1 

RITZ  VALUE  COUNTS 
:=  [Left( j,X  ) ,  Left(j,X  ),  Left(j,X  ) 

3  4  5 

* 

Step  L  L  Profile 


10  0  1  111 
11  0  1  112 
12  0  1  12  2 


15  0  1  12  2 

16  0  1  12  3 


21  1 

22  1 

23  1 

24  4 

25  4 

26  5 


2  12  3 

2  2  2  3 

2  2  3  4 

2  2  3  4 

2  2  3  4 

6  2  3  4 


9 


2 

2 

2 

3 

3 

4 
4 
4 

4 

5 
5 


Left(j,X6)] 


36 


6 


7 


2  3  4  5 
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7.  RITZ  INTERVALS 

To  sake  a  computable  theory  we  shall  replace  Ritz  values  by  Ritz 
intervals.  This  requires  some  extra  notation.  We  write 


j 


tridiag 


Pi  •••  Pj-1 

OC  i  &2  •  •  •  CtJ 

Pi  • • •  Pj-1 


Pi  >  o  ,  1  <  i  <  n, 

Pn  =  0  . 


The  normalized  eigenvectors  are  denoted  by 

i  =  1 1 .  • .  •  J  i 


T.s<j)  =  s<jVj)  , 

j  l  li 


l-£J>i.  ■ 


=  l 


The  element  of  v  is  denoted  by  v(k).  Associated  with  each  Ritz 


value  8 


(j) 


is  a  Ritz  intverval 


p  .\a 

J  i 


(j) 


(j)l  ] 


THEOREM  (C.  C.  Paige).  For  all  k  >  j  (but  k  <  n)  there  exists 
(depending  on  k  and  i)  such  that 


fl(k)  g  j(j) 


l  ~  1) . . .  i  j  » 


i 


This  useful  result  is  proved  and  discussed  in  (Parlett,  1980).  The 

radius  of  can  be  given  in  terms  of  and  * : 

X  J  J  ^ 


p..  :=  p.  Is^U) 

*ji  l 


.  »WJ'> 

'"j'1 


(j) 


Here  ♦'  denotes  the  derivative  of  ♦  .  This  interesting  result  does 
not  seem  to  occur  in  (Szego,  1939). 

One  consequence  of  this  theorem  is  a  useful  pairing  of  the  Ritz  values 
at  adjacent  steps:  Associate  to  0^^  the  value  that  lies  in 

ij*^.  If  this  rule  does  not  specify  J  (=i  or  i+1)  uniquely  then  choose 
the  that  is  closer  to 


DEFINITIONS. 


•  1^  is  isolated  if  I^J  ft  1^  =  0  =  I^j  n  1^  . 

1  l-l  l  l+l  i 

•  1^  is  marked  if  either  1^  is  isolated  or  9 €  l(^  ^ 

l  l  l  i 

and  1^  is  narked. 

In  the  event  that  1^  ^  contains  nore  than  one  j-level  Ritz 
value  we  nark  only  the  interval  whose  center  is  closer  to  0^  ^ . 


8.  THE  STANDING  HYPOTHESIS 

We  use  narked  Ritz  intervals  to  define  analogues  to  L(j)  and  R(j). 
By  convention  1^^  and  Ij]jj  are  narked. 


DEFINTIONS . 

s  A(j)  =  nax{i:  ij^  is  narked  for  all  nonnegative  k  <  i} 

s  r(j)  =  nax{i:  is  narked  for  all  nonnegative  k  <  i} 

j+l-r(j)  , 

•  The  effective  spread  (at  step  j)  is  U  l\J/ 

i=i(j)  1 

s  The  standing  hypothesis  (at  step  j)  is  that  each  narked  Ritz 
interval  contains  exactly  one  eigenvalue. 


Note  that  the  effective  spread  nay  contain  sone  interior  isolated  Ritz 
intervals.  In  contrast  to  L(j)  and  R(j),  it  is  not  the  case  that  J(j) 
and  r(j)  are  nonotone  nondecreasing  in  j.  However  we  say  that  a 
configuration  is  routine  whenever  J  and  r  are  nonotone.  In  practice 
we  use  a  violation  of  nonotonicity  to  signal  that  the  standing  hypothesis 
nay  be  false. 


DEFINTION.  The  (conputable)  point  of  discovery  - 

nin{j:  J(j)  <  J(j-l)  or  r(j)  <  r(j-l)}  . 
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Table  2  exhibits  i(j)  for  the  Dutch  example  and  the  comparison 
inner  product.  The  Ritz  intervals  are  indicated  by  horizontal  bars  in 
Figures  2  and  3. 

We  now  make  some  comments  on  the  way  in  which  monotonicity  is 

violated.  Consider  step  23  in  Figure  3.  8  and  8 *  both  belong  to 

C22)  (23) 

I  but  8  is  closer  to  the  center.  Hence  I  is  not  marked 

4  3  4 

and  J(23)  =  4.  Again  at  step  24  both  8 s  and  8 *  belong  to 

a  marked  interval.  This  time  it  is  8  that  is  closer  to  the  center 

4 

(24)  (24) 

and  so  I  is  unmarked  while  I  is  marked.  This  causes  1(24) 

3  4 

to  drop  down  to  3  and  signals  a  hidden  eigenvalue.  Similarly  at  step  25 
it  is  I<25)  that  is  unmarked  and  i(25)  =  2.  Not  until  step  36  is  Ia 
disjoint  from  1^  and  then  i(36)  leaps  up  to  6. 

The  foregoing  concepts  find  a  use  in  comparing  two  discrete  inner 
products  whose  internal  structure  is  not  known.  We  may  say  that  two 

inner  products  approximate  each  other  to  degree  j  if 

(1)  the  i(i)  and  r(i)  values  first  differ  at  i  =  j+1 

and 

(2)  each  marked  Ritz  interval  contains  the  corresponding  Ritz 
value  of  the  other  scheme. 

We  can  then  say  that  the  two  inner  products  studied  in  this  paper  approx¬ 
imate  each  other  for  polynomials  of  degree  less  than  24. 


9.  THE  I*  J  CONNECTION 
*7 

Figure  3  makes  it  clear  that  the  actual  step  at  which  the  presence  of 
a  hidden  eigenvalue  is  revealed  must  depend  on  the  actual  distribution  of 
a  number  of  eigenvalues,  not  just  those  closest  to  the  hidden  one.  One 
should  expect  a  formula  for  this  point  to  be  complicated.  Yet  it  turns 
out  that  the  necessary  global  information  is  encoded  in  a  computable 
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Table  2. 

MARKED  RITZ  INTERVAL 

COUNTS 

Step 

i 

(close  eigenvalues) 

i 

(comparison  scheme) 

7 

2 

2 

13 

3 

3 

18 

4 

4 

23 

4 

4 

— >  24 

3 

4 

25 

2 

5 

36 

6 

6 

number;  the  radius  of  the  relevant  Ritz  interval.  That  is  one  of  the 
consequences  of  the  theorem  given  below. 
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We  abandon  the  Kaniel-Paige-Saad  bounds  and  consider  instead  the 

relative  importance  of  the  contributions  of  a  close  pair  of  eigenvalues 

to  |*.|a.  The  fluctuations  of  this  contribution  during  the  stagnation 
3 

phase  of  the  Dutch  example  are  shown  in  Figures  5  and  6  by  vertical 
spikes  located  at  the  eigenvalues.  Each  A  indicates  a  Ritz  value 
and  a  circle  cuts  off  spikes  that  are  too  high.  The  sum  of  the  heights 
of  all  spikes  is  10  units. 


THEOREM.  Let  w^  =  1/n,  i  =  l,...,n,  and  let 

•  h  2—  -  Aj)/2  ^  min|x^  -  ^j+ 2  • 

•  0  :=  =  ‘4(\j  +  Xi+i)  +  eh,  -h  <  t  <  \  ,  (this  defines  i), 

.  C(J,j)  :=  {  i  xj(X,)  +  i 


•  T  .s  =  s 8,  s*s  =  1, 
J 

then 


*  (  tsKtt  H  JTSTjrr  )  +  *•'  +  °<h*>  + 

J 


Proof*  The  conditions  on  h  permit  the  use  of  Taylor  series  to 
evaluate  C(J,j).  Let  denote  *  =  0,1,2.  Then 

*j(V  =  xj +  (*j)(\ra)  +  +  0(hS)  * 

xj(Ai+i)  =  Xj  +  (x'j)(xJ+1~e)  +  ti(Xj)(xI+1-8)a  +  0(hs) 

xa.(X  )  =  [x'h(l+e)]a  fl  -  ^  h(l+e)}  +  0(h4)  }  , 

J  J  Kj 

m 

x](Xi+l)  =  fx^hCl-*)]2  {l  +  ^4  h(l-e)}  +  0(h4)  , 

J 

M 

1C  * 

C(i,j)3  =  jj  (hXj)a{l  +  -  hr  4  (3+e2)}  +  0(h4)  . 
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The  following  properties  of  tridiagonal  matrices  may  be  found  in 
(Parlett  1980,  p.129): 


s(l)s(j)Xj  =  i  * 

IxjB  =  Pr-'Pj-iPj  • 

Hence 


h 

P  .s(l)s(j) 
%) 


2 

+  C 


-  hc(3  +  *a) 


The  result  follows.  O 


In  Section  7  we  mentioned  that  the  i ^  Ritz  interval  of  diameter 

20-ls(j)l  is  guaranteed  to  contain  an  eigenvalue  and  thus 
J 

(l-*)h  <  :=  ^ls.(j)l  . 

It  is  the  factor  (h/0..)  that  controls  C(i, j)/lx  II .  The  constraint 

J 

on  e  in  the  defintion  of  h  confines  j  to  the  stagnation  phase  in 

which  the  quantity  |  will  stay  close  to  1. 

In  the  comparison  scheme  of  the  Dutch  example  (where  \=*a)t  • s ( 1 ) I 

converges  rapidly  to  >/  2/n  as  j  increases  whereas  in  the  regime 

under  study  ls(l)l  does  not  decline  to  its  limit  \/Jn  until  0 

marks  X  at  step  27.  At  the  beginning  of  stagnation  the  bound  fi . . 

2 

exceeds  h  by  nearly  an  order  of  magnitude  and  C(i,j)  is  insignificant. 
As  the  bound  drops  rapidly  towards  h  the  ratio  C(i,j)/|*.||  rises 

J 

towards  1  and,  at  that  moment,  the  minimizing  property  of  the  forces 

an  extra  Ritz  value  to  be  dispatched  towards  the  hidden  pair.  However 

the  consequent  disruption  of  the  distant  Ritz  values  is  reflected  in  a 

temporary  increase  in  fi ^  until  the  end  of  the  stagnation  phase. 

Thus  C(i, j)/Jx  I  declines  steadily  only  after  the  point  of  discovery. 

J 

In  the  Dutch  example  1^23  1 


>  10. 
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In  other  words,  the  discovery  step  is  the 
one  at  which  B  . .  has  a  local  minimum. 

r  n 


The  preceding  analysis  shows  that,  in  general,  the  discovery  step 
increases  only  logarithmically  with  a  decrease  in  h.  If  h  is 
divided  by  20  and  all  eigenvalues  except  for  and  A^+^  remain 

fixed  then  the  point  of  discovery  is  delayed  by  only  3  steps.  Here 
is  the  reason. 

This  step  may  be  characterized  as  the  solution  j  to 


C('.j) 

>*j» 


*  (=  1/3  say) 


(A) 


The  proof  of  the  preceding  theorem  shows  that  for  small  h. 


Now, 


' 2/n  h  thr 


\S<9> 


rr  (e  -  eij)) 

m*i  1,1 


(B) 


We  approximate  this  expression  by  substituting  marked  eigenvalues  for  the 
p  corresponding  Ritz  values  and  using  a  Chebyshev  polynomial  for  the 
rest  of  the  expression.  Thus, 


*3(9) 


rr  -  V-VW91 

k=l  k  j  i  p 


[Xp+l’  Xn3) 


(C) 


and  the  first  factor  is  independent  of  j.  For  (  >  1 , 

If 

T^Cf)  *  '*(£  +  ^  (a-l  )  •  Moreover,  in  most  cases,  when  j  <<  n, 

II xJ  *  Pj  (D) 

for  some  fixed  p.  For  the  Dutch  example,  p  -  0.27.  In  general,  use 
(B),(C),(D)  in  (A)  to  see  that  the  discovery  value  satisfies 

h*j  =  * 

1  2 


for  complicated  constants  * and  k^.  Thus  if  h  is  divided  by  em 
then  the  discovery  point  will  increase  by  m/in  ic^.  This  logarithmic 
dependence  is  illustrated  in  Figure  6  which  shows  the  behavior  on  the 
Dutch  example  when  the  gap  Xj-Ai  is  reduced  to  IX  of  its  original 
value. 
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produced  the  figures  which  reveal  the  patterns  so  clearly. 
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Figure  6.  RITZ  INTERVAL  DEVELOPMENT 
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